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I. INTRODUCTION 
According to the United Nations 





Development 


based on the definition of quality of education proposed by the United 
Nations Educational, Scientific, and Cultural Organization (UNESCO). 
The problem that arises in the construction of an indicator of educational 
quality is that its components seek to measure subjective values. This 
makes it necessary to introduce imprecision in the analysis to reduce 
evaluation errors. Thus, in addition to the comprehensiveness of 
dimensions, an important aspect of this study is the use of the Composition 
of Probabilistic Preferences (CPP), a multicriteria method to support the 
manager in the decision-making process that considers imprecision from 
the beginning. Finally, it is worth highlighting the possibility of 
application of the proposed indicator, using the variables of the quality 
panel proposed by the United Nations Development Program (UNDP) 
and the incorporation of this index in the Human Development Index 
(HDI). 


motivating this research, which will also have the objective 
of incorporating a probabilistic component in the evaluation 
of the quality of education. 


Programme (UNDP), the Human Development Index 
(HDI), created by the Pakistani economist Mahbub ul Haq 
with the collaboration of the Indian Nobel laureate in 
Economics Amartya Sen, is intended to contrast with a 
widely used indicator, the Gross Domestic Product (GDP), 
which considers only the economic dimension of 
development. The HDI has been measured annually since 
1990, incorporating some changes over the years. The HDI 
is composed of three dimensions: income, health, and 
education. 


The HDI uses school attendance in the education 
dimension, but this measure may not be enough to evaluate 
the quality of education provided in each country. This 
justifies the need to insert a quality indicator for education, 
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The most widely used definition of quality in education 
is that of citizens who can read and interpret in their mother 
tongue and have the skills to apply mathematical logic 
reasoning in their daily lives [28]. In this sense, there are 
tests such as the Programme for International Student 
Assessment (PISA) and, at the national level, the 
BrazilianSystem for Basic Education Assessment (SAEB), 
the BrazilianHigh School Exam (ENEM) and the 
BrazilianStudent Performance Exam (ENADE), focused on 
higher education. 


But can these evaluations alone provide a good picture 
of quality education? This work proposes to look beyond 
these evaluations, considering other factors that can have an 
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impact on this quality. Such as the teacher's qualification 
and how the school environment with internet access 
broadens the classroom education. The ratio of the number 
of students per teacher is also an important factor, because 
its reduction enables teaching with a more individualized 
look. 


The present research aims to build a_ probabilistic 
indicator of quality of education more comprehensive than 
the existing ones from an interdisciplinary approach, using 
the structural bases of probabilistic modeling, which in this 
work will be the Composition of Probabilistic Preferences 
(CPP), developed by Annibal P. Sant'Anna since 2001. The 
incorporation of this indicator of the quality of education, 
contemplating the dimensions of pedagogical, cultural, 
socialand financial quality, allows to amplify the relevance 
of the HDI. 


The concept of quality of education used in this work 
may change over time. Thus, we can reformulate the criteria 
according to the new needs of society. However, he 
methodology used to construct the index can continue to be 
used. 


In a search in the literature, no probabilistic index of the 
quality of education was found with the breadth of the 
developed index, contemplating the pedagogical, cultural, 
social and financial dimensions. Most of the indexes are not 
probabilistic and contemplate only the pedagogical 
dimension. 


The structure of this research is divided into five 
sections detailed below. 


Section 1 is composed of the introduction to the theme, 
through the contextualization of the problem, justifying the 
need to create an index of quality of education that 
comprises other criteria and that can be aggregated to the 
HDI. 


Section 2 describes the criteria that are being used to 
construct the CPP_QEdu indicator. 


Section 3 presents the modeling and methodology used 
to construct the indicator. 


Section 4 shows the calculation for extending the HDI 
by incorporating the proposed education quality index. 


Section 5 presents the analysis of the results of the 
application of the developed index to the states and 
municipalities of Brazil. 


I. DESCRIPTION OF THE CPP_QEdu CRITERIA 


As seen earlier, the quality of education can be 
understood from various perspectives. Based on this 
diversity, UNESCO points out four dimensions that make 
up quality: pedagogical, cultural, social, and financial. 
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The pedagogical dimension is the effective fulfillment 
of the curriculum. For this, we can use evaluation tools like 
PISA, SAEB, ENEM, and ENADE. 


In the cultural dimension, curricula need to be in 
accordance with the diverse regional cultures, and for this 
the teacher needs to have adequate training. The well 
qualified teacher who uses an appropriate curriculum 
develops strategies to make school more interesting and, 
consequently, reduces dropouts and increases school 
attendance. Thus, the teacher's training will be the criterion 
used in the cultural perspective. 


From the social perspective, UNESCO associates the 
quality of education with its contribution to equity. In this 
sense, it checks whether the school is, in a democratic way, 
giving the opportunity for students of the same age group to 
have access to the same content, as anywhere on the planet. 
The use of the Internet in the teaching-learning process 
provides equal access to information. For this reason, the 
percentage of schools with Internet access will be used as 
an indicator to represent the social indicator. 


From the economic point of view, quality refers to 
efficiency in the use of resources allocated to education. To 
represent this criterion, the number of students per teacher 
will be used. In this item, there is a counterpoint: more 
students per teacher would be less costly, but would have 
lesserbenefit. Thus, reducing the number of students per 
class characterizes the quality in the use of resources 
[59][60]. 


The United Nations Development Programme (UNDP) 
Human Development Report 2019[56], which has a central 
theme of human development inequalities in the 21st 
century, elaborates a quality of human development 
dashboard with three quality indicators. This Quality of 
Human Development dashboard is a table containing a 
selection of 14 indicators associated with the quality of 
health, education, and income. The three health quality 
indicators are: health expectancy lost, number of doctors, 
and number of hospital beds. The seven education quality 
indicators are: pupil-teacher ratios in elementary school, 
primary teachers trained to teach, proportion of elementary 
school with Internet access, proportion of secondary schools 
with Internet access, and the Programme for International 
Student Assessment (PISA) results in math, reading, and 
science. The four quality of life indicators are: the 
proportion of vulnerable jobs, the proportion of the rural 
population with access to electricity, the proportion of the 
population using improved drinking water sources, and the 
proportion of the population using improved sanitation 
facilities. 


Based on the quality of education indicators used by 
UNDP in the Quality of Human Development panel and the 
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Quality of Education dimensions pointed out by UNESCO, 
seven criteria were used to compose the CPP_Qedu, 
covering these four dimensions. The criteria are the number 
of students per teacher, percentage of schools with Internet 
access, percentage of teachers with higher education, SAEB 
Mathematics score for the Elementary School - Early Years, 
SAEB Portuguese Language score for the Elementary 
School - Early Years, SAEB Mathematics score for the 
Elementary School - Final Years, and SAEB Portuguese 
Language score for the Elementary School - Final Years. 


In the UNDP indicator, the pupil-teacher ratio in 
elementary school is used, while in the CPP Qedu the ratio 
between the number of pupils per teacher in primary and 
secondary education in Brazil is used as an equivalent 
criterion. The second criterion used is the percentage of 
teachers with higher education in primary and secondary 
schools in Brazil, corresponding to the ratio of primary 
teachers trained to teach. The third criterion, percentage of 
primary and secondary schools in Brazil with Internet 
access, is analogous to two indicators: the proportion of 
elementary school with Internet access and the proportion 
of secondary schools with Internet access. The remaining 
four criteria are the SAEB (Basic Education Evaluation 
System) scores for Mathematics and Portuguese Language 
of the two phases of elementary school. This assessment 
developed by the Brazilian government is similar to the 
Programme for International Student Assessment (PISA) in 
mathematics and reading. 


HI. CPP Qedu DATA MODELING 


The modeling will use Composition of Probabilistic 
Preferences(CPP) [45], based on the application of 
probability theory im decision support. CPP is a 
methodology developed to take into account, in the 
composition of multiple criteria, the presence of 
imprecision in preference evaluations. Inaccuracy is 
inherent in the subjectivity and errors of evaluation in 
decision making processes. Thus, a random component is 
recognized for the evaluation of each alternative under each 
criterion. 


CPP deals with rules for combining evaluations by 
different criteria or different experts in terms of choice 
probabilities. It starts bythe evaluation of the preference of 
each alternative by the probability of that alternative being 
chosen among the others. To arrive at this result, the 
preference given by the value of a performance attribute is 
not treated as a precise and definite measure of preference, 
but as a random variable. 


Thus, a probability distribution is associated with each 
measurement, in a manner similar to that employed in fuzzy 
sets theory [64] to replace exact numbers with measures of 
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relevance to the points in the intervals around them. The 
initial exact measurements are considered as parameters for 
locating probability distributions of the possible values that 
in other evaluations under similar circumstances would be 
assigned to the same option. This replacement of exact 
measurements with probability distributions constitutes the 
first step of CPP, according to Sant Anna (2015)[46]. 


The second step refers to the choice of a probability 
distribution that is identified, or even approximately 
assumed, to be characteristic of the disturbances affecting 
the measurements, with the exact values as location 
parameters. 


The third step involves two computations. The 
probabilities that the i-th alternative is superior (M;;) and 
inferior (m;;) to the others are calculated for each j-th 


criterion. According to equations (1) and (2), 


Mi; =| | | Fj) fx, i) dxi (1) 


Dx 


my = | 
D 


where Fx, represents the cumulative distribution 


il Ji 


[| (1 = Fe (65) fx xi) dx; (2) 


Xi | j+i 


function (cdf), fy, the probability density function (pdf) and 
Dy, the domain of the random variable xj, as in Sant'Anna 


et al (2012)[42]. The calculations are performed for each 
alternative under each criterion. 


In the fourth step, the joint probabilities M;; and mj; 
are composed into global preference scores under different 
decision points of view. Compositions based on M;; are 
called progressive because they are associated with gain 
maximization. Compositions based on mi;are called 
conservative, because they are associated with loss 
minimization. On the other hand, maximizing M;; and 
minimizing mj,; in all criteria are called pessimistic 
compositions, while optimistic ones are based on 
satisfaction in only at least one criterion with the maximum 
or minimum preference. Four points of view then emerge. 


The Progressive-Pessimistic (PP) viewpoint calculates 
preference by the probability of being the best according to 
all the criteria considered. The calculation is performed 
according to the hypotheses of independence and maximum 
dependence, which portray the extremes of correlations 
between variables, respectively, by Equations (3) and (4). 


pp, =| [My (3) 


PP, min M;; (4) 
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The Progressive-Optimistic (PO) viewpoint uses the 
probability of being the best according to at least one of the 
criteria considered, calculated according to Equations (5) 
and (6). 


PO; = max Mi; (6) 


The Conservative-Pessimistic (CP) viewpoint uses the 
probability of not being the worst in all the criteria 
consideredand is calculated from Equations (7) and (8). 


CP; = 1 — max mij (8) 


Finally, the Conservative-Optimistic (CO) viewpoint uses 
the probability of not being the worst in at least one of the 
criteria considered, and is calculated according to Equations 
(9) and (10). 


The problem that arises in the construction of an 
indicator of quality of education is that its components seek 
to measure subjective values, which are affected by political 
and methodological problems. It is also necessary to 
consider the aspect of sustainability in education, 
contemplating the balance between the dimensions of 
quality of education proposed in this work. All aspects 
considered are equally important in the composition of this 
indicator. 


The initial transformation into preference probabilities 
solves the problem of choice of scales, transforming the 
measures of each dimension into probabilities of reaching a 
frontier of better or worse performance in that dimension. 
And it contributes to improving the efficiency of decisions, 
since it improves the level of precision with respect to 
quality. 


In this paper, the conservative-optimistic probabilistic 
composition (CO) approach will be used: in which the 
alternative is evaluated by the probability that it is no worse 
than the sample by at least one of the criteria. Conservative- 
Optimistic composition seeks the goal of escaping the large 
social differences that exist in Brazil and in the world in the 
particular aspects considered, while hoping that the 
momentum generated in some dimension will have the 
effect of causing advances in others. 
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IV. HDI EXPANSION 
4.1 HDI calculation 


The Human Development Index (HDI) is a summary of 
measures in three dimensions, which are the key to human 
development. A long and healthy life in the health 
dimension, access to knowledge represented in the 
education dimension, and a decent economic standard, 
which is the income dimension. The HDI is the geometric 
mean of the normalized indices for each of the three 
dimensions. 


As of 2010, a new calculation method was developed to 
construct the index, according to Technical Note 1 of the 
HDI report. In this new calculation, UNDP continues to 
combine the three dimensions, health, education and 
income, but uses new variables in some dimensions: 


In the health dimension, it uses life expectancy at birth 
in the locality in question. The variable is the same, but there 
was a change in the calculation, because there was a change 
in life expectancy in the world. 


To construct the life expectancy index (IEV), we use 
equation (11): 
EV-20 
85-20 





(EV) = (11) 


Where EV is life expectancy at birth, IEV is 1 when life 
expectancy at birth is equal to or greater than 85 years and 
O when life expectancy at birth is equal to or less than 20 
years. 


In the Education dimension, today it uses the arithmetic 
average of the observed number of years of schooling and 
the expected number of years of schooling for that location, 
truncated at 15 and 18, respectively. Previously, the literacy 
rate and the schooling rate were used. As literacy is no 
longer a problem in most countries, this value has become 
less significant. 


To construct the education index (IE), we do the 
following calculations: 


First, we construct the index of average years of study 
(IAME), that is, years that a person aged 25 or older spent 
in formal education, equation (12). 

AME 


(IAME) = —— (12) 


where 15 is the maximum projected in this indicator for 
2025. 


Next, we construct the expected years of schooling 
index (IAEE), that is, the expected number of years of 
schooling a child will achieve at birth, equation (13). 

AEE 


(IAEE) = — (13) 
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where 18 years is the equivalent of obtaining a master's 
degree in most countries. 


The education index (IE) is calculated by taking the 
simple arithmetic average of the two indexes (IAME) and 
(IAEE), represented in equation (14). 


(IAME)+(IAEE) 


(IE) =“ 


(14) 


In the income dimension, Gross National Income (GNI) 
per capita (GNIpc) is used, with a new calculation. 


In(GNIpc)-In(100) 


UR) = In(75000)-In(100) 


(15) 


where IR is 1 when GNI per capita is equal to or greater 
than $75000 and O when GNI per capita is $100. GNI per 
capita is the dollar value of a country's final income in a 
year, divided by its population. 


Finally, the HDI is calculated by the geometric mean of 
the indices, from equations (11), (14) and (15). 


HDI = 3/(IEV). (IE). (IR) (16) 


To extend the HDI with the education quality index 
proposed in this paper, at first, the 7 education quality 
indicators from the human development quality dashboard 
proposed by UNDP in 2018 are used. 


4.2 Calculation of CPP QEdu with data from 
UNDP's Human Development Quality Panel 


In a first step, the missing data were replaced by the 
median of the human development quality scoreboard 
proposed by UNDP in 2018, Table 1. This includes most 
countries as far as PISA data is concerned. 


Table 1: Human Development Quality Scoreboard - 
UNDP - 2018 


https://docs.google.com/document/d/1 eiKf9 E2huxDW 
LX3K4EOUdEfJcCMhCFrAAFwKezGwPck/edit?usp=sh 


. 


arin? 





We use the seven criteria from the panel, Table 1, to 
compose the index. These data are normalized, passing the 
minimum to zero and the maximum to one. After this 
standardization of the data, we randomize them, using these 
values as the mode of a triangular distribution with extremes 
zero and one. These values are then used to calculate the 
probabilities of maximizing (Mij) and minimizing (mij) the 
data. Assuming independence, we use the following 
combinations of maximization and minimization to find the 
four profiles: 
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Conservative/Optimistic: COi = 1-[ [jmij, (17) 
Conservative/Pessimistic: CPi = | [j(1-mij), (18) 
Progressive/Optimistic: POi = 1-| [j(1-Mj)), (19) 
Progressive/Pessimistic: PPi = | ]jMyy. (20) 


where | | denotes the product operator. 


The comparison rules adopted are the conservative ones, 
which are based on the distances to the worst performing 
extremes, equations (17) and (18). The justification is that 
the goal of the analysis is to identify the countries that need 
more help with respect to education quality. The optimistic 
conservative composition and the pessimistic conservative 
composition are analyzed. 


In the optimistic approach, the country is evaluated 
using the probability that it is not the worst in the sample by 
at least one of the criteria. This assumption captures the 
hope of escaping underdevelopment in any of the particular 
characteristics considered by generating favorable 
momentum to advance in the other dimensions. This 
optimistic assumption, combined with an assumption of 
statistical independence, results in a score that is calculated 
using the complement of the product of the probabilities of 
showing the worst assessment in each dimension. In the 
pessimistic approach, the country is evaluated using the 
probability of not being the worst in all criteria. As there 
was no significant difference in the ranking, Table 2, we 
opted to present only the result of applying the 
Conservative/Optimistic approach to compose the HDI. 


In Table 2, the following countries were excluded: 
Armenia, Gabon, Haiti, Iraq, Libya, Palau, Sudan, Syrian 
Arab Republic, Timor-Leste, Turkmenistan, Venezuela, 
which did not have at least one data in the UNDP quality 
panel. For this reason, the HDI ranking was recalculated 
without these countries to compare with the IDH_QEdu. 


4.3 CalculationofIDH QEdu 


To extend the HDI education index, we use the index 
CPP QEdu(CO), which will be introduced in the arithmetic 
mean IE of the HDI, equation (21), introducing the quality 
of education in the indicator, equation (22). 


(IAME) + (IAEE) + CPP_QEdu(CO) 


(EQ) = ; 


(21) 





IDH QEdu = \j (IEV).(IEQ). (IR) 


(22) 


In Table 2, we present the results comparing the HDI 
ranks with the extended HDI (IDH QEdu), removing the 11 
countries that did not have quality data. 
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Table 2: Expanded HDI ranking - 2018, compared to the 
HDI. 


https://docs. google.com/document/d/lkx2JeulYE2m2jFr 
OtDRLFDhHUiWKXh&8&DmxSElIfxtT0/edit 2usp=sharing 





The quality indicator did not impact, significantly, the 
rank of the first quartile countries; there were small changes 
in positions. But it had a positive impact on some countries 
in the second quartile, such as Panama, which rose 16 
positions in the ranking. Costa Rica also had a significant 
position change. However, the countries that were impacted 
the most were those in the 4th quartile, many of which had 
significant changes in their positions for better or worse. 
Thus, the indicator is able to identify the countries that have 
been seeking improvements in the quality of education; a 
change in some criterion can impact the change in position. 


V. ANALYSIS OF THE RESULTS 
5.1 Analysis of the Brazilian States 


For a first analysis of the adequacy of CPP_Qedu with 
SAEB data, the data of Brazilian states from the year 2017 
were used, Table 3: 


Table 3: Indicators per BrazilianState 





https:/docs.google.com/document/d/LiA5y8§soOP2PVk 
OZyINEldTl04agJT917 Uu6Ow 1 scs6g/edit ?usp=sharing 


To analyze the results obtained with thesedata, we began 
with a comparison of the 2017 Municipal Human 
Development Index (MHDI) results by states with the 
results of the Conservative-Optimistic axis of the CPP. We 
started with the MHDI Education, a synthetic index of the 
education dimension that is part of the MHDI composition, 
which is composed of three dimensions: longevity, 
education, and income. This indicator is obtained through 
the geometric average of the subindex of children and young 
people's school attendance, with a weight of 2/3, and the 
subindex of the adult population's schooling, with a weight 
of 1/3. The CPP-QEdu has seven criteria, based on the new 
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education quality indicator, these results are described in 
Table 4. 


In this first analysis, three states show significant 
differences in the comparison. The state of Roraima appears 
in fourth place with the MHDI-Education methodology, 
demonstrating an excellent job in getting children into 
school and improving the schooling level of the adult 
population. However, this state does not seem to have given 
much importance to the quality of the education offered, 
since with the CPP_QEdu it appears in 20th position, 
highlighting the importance of analyzing the quality criteria 
in the human development indexes. 


Table 4: Comparison of the Indexes per Brazilian State 


https://docs. google.com/document/d/1A Kr2pA6B614QE 
amt2gzKV7qDKFAOm50Sn0TuVTBJigs/edit?usp=shari 
ng 





Looking at Ceará and Mato Grosso do Sul, we observe 
that these states still need to improve the inclusion of all 
children in school and increase the schooling of the adult 
population, as they appear in the 14th and 15th positions in 
the MHDI Education ranking.However,it can be seen that 
they are moving in this direction, as they are in a median 
position in this ranking. When we examine the results of the 
CPP_QEdu, which focuses on the quality of education, we 
see that, although this inclusion is happening more slowly, 
these states appear in the 5th and 6th positions in this 
ranking, which indicates that these states are investing in the 
quality of education offered. 


In Figure 1, in the radar graph that compares the 
rankings of the two methodologies, the Education MHDI 
ranking with the CPP_QEdu ranking from the conservative- 
optimistic point of view, a very distinct design can be seen 
in the methodologies, that is, the Education HDI ranking is 
very different from the CPP_Qedu ranking in most states, 
confirming the importance of using indicators that evaluate 
the quality of the education offered and not only how long 
a student has been in school. There is no point in staying a 
long time in a school that offers a precarious teaching- 
learning process. 
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— RANK IDHM 
Edu 


MHDI EDU vs. CPP QEdu(CO) 


em RANK 
CPP_Qedu(CO) 


; Rondônia 
a A 
Distrito FederB0 C Amazonas 
Mato Grosso 2 Roraima 
Mato Grosso... 20 Para 
Rio Grande do... 15 Amapá 
Santa Catarina Tocantins 
Paraná Maranhão 
São Paulo Piauí 
Rio de Janeiro Ceará 
Espírito Santo Rio Grande do... 
Minas Gerais ~ Paraíba 
Babifcipe Alagoggnambuco 


Fig.l: Spider graph comparing the results of the 
MHDI EDU with the CPP_QEdu 


Source: Authors 


To verify the influence of SAEB scores on the 
CPP QEdu indicator, we compared the performance of the 
CPP QEdu indicator with the CPP SAEB. The 
CPP SAEÉEB was calculated with the four SAEB score 
criteria per state (SAEB Math score for Primary School 
Mathematics, SAEB Portuguese Language score for 
Primary School Mathematics, SAEB Math score for 
Primary School Mathematics, Final Years, and SAEB 
Portuguese Language score for Primary School Portuguese 
Language, Final Years), as shown in Table 5. 


It can be seen that the states of São Paulo and Minas 
Gerais were strongly influenced by the SAEB scores. 
However, when the results of Rio de Janeiro and Rio Grande 
do Sul are analyzed, it is observed that the SAEB score had 
little influence on the CPP_QEdu ranking, so the other three 
criteria had a strong influence on the result. This analysis is 
important to show that the CPP QEdu index is not focused 
only on content quality and that the other criteria are also 
influencing the ranking. 


Observing, in Figure 2, the radar chart, which compares 
the CPP_QEdu with the CPP_SAEB with the same 
conservative-optimistic point of view, there is a very close 
approximation of the two curves. In this view, there is a 
strong influence of SAEB scores on the CPP_QEdu 
indicator. 
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Table 5: Comparison of CPP_QEdu with the SAEB 
performance index 


https://docs. google.com/document/d/IvyScEYOh7WJBn 
nT vKS8UuRh-zF_itRJ1tBd7oy7K9VLc/edit ?usp=sharin 





CPP_CO vs. CPP_SAEB — RANKSAEB 
— RANK CO 
o Rondônia 
Distrito Federgo BO acta 
Mato Grosso 25 Roraima 
Mato Grosso... 20 Pará 
Rio Grande do... 15 Amapá 
Santa Catarina Tocantins 
Paraná Maranhão 
São Paulo Piauí 
Rio de Janeiro Ceará 
Espírito Santo Rio Grande do... 
Minas Gerais ~ Paraíba 
Babifcipe — Alagoggnambuco 


Fig.2: Spider graph comparing CPP SAEB results 
with CPP QEdu (CO) 


Source: Authors 


Although the context of the problem better fits the 
Conservative-Optimistic point of view, it is considered 
relevant to verify the sensitivity of the model, comparing it 
with the result of the application of the compositions by the 
other points of view, Table 6. 


In this sensitivity analysis, we verify the consistency of 
the state of Santa Catarina which, under different points of 
view, remains in first place in the ranking, that is, the state 
with the best quality of education in the country. In this 
same analysis, we find the state of Pará among the worst in 
Brazil from all points of view. 
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Table 6: Comparison of the Four Viewpoints of 
CPP_QEdu 


https://docs.google.com/document/d/1A- 
wsP2VPe24zf 22yW-5Rc- 
TISKymN8&VTwh8xBpsongS/edit ?usp=sharin 


Model Sensitivity in Different Viewpoints 





RANK - CP 
— RANK - CO 
RANK - PO 
- Rondônia ms RANK- PP 
Go 900a! ACTE mazoa 
Mato Grosso Roraima 
Mato... 1900ral Para 





Rio Grande... Amapá 
190 á 
Santa... Tocantins 
Paraná 190 Maranhão 
São Paulo Piauí 
Rio de... Ceará 
Espírito... Rio Grande... 
Minas Gerais ~ Paraíba 
Babidgipe Alagbggnambuco 


Figure 3: Spider graph comparing the Four Viewpoints of 
CPP_QEdu 


Source: Authors 


At the same time, the states of Rio de Janeiro and Ceara 
behave much better in the conservative profiles, where 
losses are minimized. As most of the criteria are close to the 
maximum, the states of São Paulo and the Federal District 
behave much better in the progressive profile. 


Figure 3 shows the behavior of the rankings in the four 
approaches analyzed. It can be seen that the four approaches 
have similar results, with the distortions previously 
analyzed. 


5.2 AnalysisofBrazil'sMunicipalities 


An analysis of the municipalities was also carried out. 
The same variables were used: number of students per 
teacher, percentage of schools with Internet access, 
percentage of teachers with higher education, SAEB 
Mathematics score for the Elementary Beginning Years, 
SAEB Portuguese Language score for the Elementary 
Beginning Years, SAEB Mathematics score for the 
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Elementary End Years, and SAEB Portuguese Language 
score for the Elementary End Years. 


As for missing data, to be expected in a dataset of all 
Brazilian municipalities, including representation from 
municipalities far away from urban centers, it is interesting 
to note that few municipalities had missing data. To impute 
values, we used the following procedure. First, the values of 
each variable were rescaled to the interval [0,1]. Then, the 
missing values were imputed with the median of the values 
of the other variables in the municipality. 


Two analyses were conducted keeping the data for the 
explanatory variables of performance quality fixed at their 
2019 values. For the variables assessing the results of 
curriculum delivery effectively, 1.e., the SAEB variables, 
2017 data were considered in the first analysis and 2019 
data in the second. 


To avoid the effects of data imprecision from variables 
representing smaller presences, the comparison was limited 
to the municipalities in the top quartile in the ordering of 
municipalities by size, represented by the number of 
students enrolled in 2019. In this quartile are municipalities 
with 6180 students or more. Of the total 5570 municipalities 
in the database, the top quartile consists of 1393 
municipalities. 


To provide an objective comparison of the differences, 
a selection of data from extreme municipalities from the 
extreme states is presented here, that is, the states of Santa 
Catarina, Parana and Rio de Janeiro on one side, and the 
states of Amapá, Maranhão and Pará on the other. 


Initially, the differences in the size of these states should 
be noted. Amapá has only 6 municipalities in the analyzed 
quartile, Maranhão 96 and Pará 102. On the other side, 
Santa Catarina has 52 municipalities in the upper quartile, 
Parana has 71 and Rio de Janeiro 54. 


Table 7: CPP_QEdu Resources and Rank Values of 
Municipalities in the Extremes 


https://docs.google.com/document/d/1AslEHAeyaLnksM 
DIGSIZMTtOFEmGKEpsOxkS Vc9o0JcE/edit ?usp=sharin 


& 





In Table 7, one can observe the expected distance 
between the values corresponding to municipalities at the 
two extremes, but one can also highlight the variation 
between the availability of educational resources within the 
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states. Thus, in Macapá, capital of Amapá, the percentage 
of teachers without higher education is much lower than in 
the other municipalities of the state, while the proportion of 
schools with Internet resources is much higher. In 
Maranhão and Pará, the internal variability comes from the 
fact that, while municipalities like Santa Quitéria do 
Maranhão and Porto de Moz rank last among the 
municipalities in all the states, the municipalities in 
Maranhão and Pará, as a whole, rank better than those in 
Amapa, Table 5. 


Tables 8 and 9 allow us to compare, respectively, the 
performances of the municipalities in the lower end and the 
upper end listed in Table 7. The values in these tables are, 
respectively, the probabilities of presenting the worst and 
the best score in each of the tests considered in the index. 
These probabilities allow the comparison to be free from the 
effect of the possible variation in the difficulty of the tests 
from one year to another. 


Table 8: Odds of Lowest Scoring Municipalities in the 
Worst-Ranked States 


https://docs.google.com/document/d/1sKG7xItLul GpH; 
ogGViNScfCROxuUu01GtBQOnfZcr9 g/edit ?usp=sharing 





Table 9: Odds of Highest Score of Top-Ranked 
Municipalities in Top-Ranked States 


https://docs. google.com/document/d/lklvcvc3mnO0lYe 
gXNx9sa-e-OC-NI9NYKGL77TRC-A/edit ?usp=sharing 





From the point of view of the values recorded in these 
tables, the highlight is the agreement between the results of 
the two years. 


It should also be noted that these probabilities are 
calculated in R with remarkable precision despite the large 
size of the data set, 1393 municipalities. Even with the 
separation into quartiles, we still have over 1000 (almost 
1400 in the top quartile) municipalities in the same group. 
This comparison of such large samples, however, proved 
feasible, even though CPP compared each alternative with 
all the others. 
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VI. CONCLUSION 


The present research achieved the goal of constructing a 
probabilistic indicator of quality of education that is more 
comprehensive than existing ones. From an 
interdisciplinary approach, using the structural basis of the 
probabilistic modeling offered by CPP. To extend the HDI, 
the developed index, CPP_QEdu, was incorporated. The 
incorporation of this education quality indicator 
contemplates the dimensions of quality, pedagogical, 
cultural, social, and financial, proposed by UNESCO, 
extends the relevance of the HDI. 


The variables representing these dimensions were based 
on the human development quality dashboard proposed by 
UNDP in 2018. That body developed the dashboard, but did 
not develop a measure that could be incorporated into the 
HDI, as was done in this research. These variables that were 
used to construct the indicator per country are adequate, as 
they contemplate the dimensions of quality proposed by 
UNESCO. 


In this work, a CPP _QEdu indicator was also 
constructed for the states and municipalities of Brazil, 
where the Ministry of Education works with a quality 
indicator, the IDEB, which, however, only considers the 
pedagogical and social dimensions. To construct the 
CPP_QEdu at the national level, variables equivalent to 
those in the UNDP panel were used. All variables used to 
compose the index cover all basic education in the states or 
municipalities, without restricting the segment of education. 
The PISA scores (Mathematics, Portuguese Language and 
Science) were substituted by SAEB scores (Mathematics, 
Portuguese Language) in two segments, for greater national 
coverage. The indicator constructed is more comprehensive 
than the IDEB and can contribute to the construction of 
better public policies that provide higher quality education 
in the country. 


The dimensions of quality of education that were 
incorporated into the CPP_QEdu were based on the 
dimensions of quality proposed by UNESCO and on the 
variables used in the panel on quality of human 
development developed by the UNDP. At the beginning of 
the research, other variables were evaluated to compose the 
index, but as the research evolved, we concluded that these 
dimensions were already broad enough. The standardized 
tests in the case of PISA and SAEB to compose the 
pedagogical dimension are appropriate, because they 
evaluate the knowledge acquired by the students. In the ratio 
of students per teacher to compose the financial dimension, 
the balance of this ratio ratifies the efficiency in the use of 
resources directed to education. The teacher needs to have 
adequate knowledge to develop a curriculum in accordance 
with cultural differences, making the school more attractive. 
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In this way, the teacher's training is propitious to compose 
the cultural dimension. And the access to the Internet to 
compose the social dimension 1s justified by the equality in 
the reach of information. 


The previously existing indexes do not have 
probabilistic features and, for the most part, evaluate the 
quality of education only from the pedagogical point of 
view. Only the IDEB, a national indicator, is calculated 
from crossing of student performance data with approval 
rate, evaluating two dimensions. 


As future work, it is suggested to consider the 
importance of introducing a variable that evaluates gender 
inequalities in access to education, in Brazil and in the 
world, since these differences are relevant in several places. 
Another analysis can be done by separating the 
municipalities that are in rural regions from the essentially 
urban ones; the evaluation was done by the number of 
students attended in the region, but there was no separation 
by type of region. The approval rate and the grade/age 
distortion are variables that can be introduced in the index, 
because, when composed with the standardized tests, they 
help to identify when students are approved without having 
acquired knowledge. An important study at the moment 
would be an analysis of the impact of the COVID 19 
pandemic on the quality of education, as well as including 
new educational modalities as criteria. 
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